Recently, many international standards for still image and video compression such as the ITU-T H261, H263, H264 and the ISO JPEG, MPEG-1, MPEG-2 standards have mainly proposed the block based Discrete Cosine Transform (DCT) as a possible compression technique.
At low and moderate bit rates, block-based DCT coding artifacts become perceptible. Such artifacts are known as mosquito noise or ringing noise occurring around edges within an image or near a smooth zone as well as the blocking effect. For still pictures or still parts of image, the blocking effect is dominant and visible in smooth regions. For dynamic video sequences and in high resolution large screen display, mosquito noise can become evident for the human vision system (HVS).
There are many existing techniques for blocking effect reduction. In H. Reeve and J. Lim, “Reduction of blocking effects in image coding”, Optical Engineering, vol. 23, January/February 1984, pp. 34-37, the authors teach the systematical use of low-pass filters applied at block boundary. Low pass filtering is utilized also in U.S. Pat. No. 5,850,294 to Apostolopoulos et al. for blocking artifact reduction purposes. However, the blocks that potentially exhibit block artifacts are detected in the DCT domain and low-pass filtering is applied only for the distorted blocks. In B. Ramamurthi and A. Gersho, “Nonlinear Space-variant post processing of block coded images”, IEEE Transactions on Acoustics, Speech and Signal Processing, vol. ASSP-34, October 1986, pp. 1258-1268, the proposed adaptive filtering is based on the detection of edge orientation at each block boundary pixel. Many authors, as in, for instance, A. Zakhor, “Iterative Procedure for Reduction of Blocking Effects in Transform Image Coding”, IEEE Transactions on Circuits and Systems for Video Technology, vol. 2, No. 1, March 1992, pp. 91-95, have proposed various multi-pass procedure techniques for this purpose. The iterative techniques can provide potentially a higher performance than the non-iterative ones, but are less attractive for real time processing.
Moreover, in existing techniques, the block localization is assumed to be known. This assumption is valid when the block correction is applied directly after the compression decoder. However, for home theater applications, the artifact correction can be everywhere from the decoder output to the final displayed image. In such situation, the considered image can be partially cropped or modified by various manipulations or even by analog conversion. The block position can thus be changed in respect to the image borders.
For mosquito noise artifact reduction (MNR), in U.S. Pat. No. 5,610,729, Nakajima teaches an estimation of block mean noise using the quantization step and the I, P, B coding mode when these data are available from the compressed bit stream. Nakajima teaches also the use of the well-known Minimum Mean Square Error (MMSE) filter proposed originally by J. S. Lee in “Digital image enhancement and noise filtering by use of local statistics”, IEEE Transactions on PAMI-2, March 1980, pp. 165-168, for artifact reduction. However, in many applications, the quantization step or the coding mode is not necessary known or accessible. Moreover, while the Minimum Mean Square Error filter is efficient for edge reservation, it is not necessary for noise reduction near an edge. Mosquito Noise is a compression noise around edges.
In U.S. Pat. No. 5,754,699, Sugahara discloses a similar approach by using block quantization step size information for noise power estimation and an empiric coring technique for artifact filtering.
Also for MNR, in U.S. Pat. No. 5,850,294, Apostolopoulos et al. disclose a filtering on the true non-edge pixels within blocks containing edge pixels rather than smoothing the edge pixels, to avoid eventual blur and picture sharpness loss due to true edge filtering. However, the filtering technique for non-edge pixels is not clearly specified.
In a same manner, in U.S. Pat. No. 5,852,475, Gupta et al. apply separable low pass filters only on portions of an image that are not part of an edge and are not part of areas of texture or fine detail. The proposed post processor contains also a temporal digital noise reduction unit for noise flicker reduction and a look up table based shape adaptive window for reliable filtering on edge and texture. For the chrominance signals Gupta et al. teach the use of simple low pass filtering. U.S. Pat. No. 5,920,356 to Smita et al. is an ameliorated version of U.S. Pat. No. 5,852,475 in which the filtering is controlled by a coding parameter of the replenished macro-blocks.
In U.S. Pat. No. 6,064,776 to Kikuchi et al., in a similar manner, a given block is classified according to whether it is considered part of a flat domain or not. If a block is considered as part of a flat domain, block pixel correction is then given by an AC component prediction technique.
In U.S. Pat. No. 6,188,799, Tan et al. teach the use of separable low-pass filtering, when block boundaries are located, for a serial reduction of blocking effect and then, mosquito noise. For detected blocking effect, the pixels are firstly corrected by a proposed modified version of bilinear interpolation and secondly, by a mean value of homogenous neighboring pixels within the quantization step size.
In U.S. Pat. No. 6,304,678 B1, Yang et al. teach the use of iterative pixel clustering technique in a sliding window and the artifact correction mainly based on maximum likelihood estimation. There is no discussion about real-time processing.
In PCT Application No PCT/CA02/00887, an adaptive spatial MNR has been proposed. The temporal dimension important for some artifact flickering is not considered by the application. The blocking detection and correction are not also considered.